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ABSTRACT 

People with hearing disability find it difficult to access to infonnation and communication in public places. According to 
this fact, it is considered the possibility to design a communication system based on the Spanish Sign Language (SSL), 
which helps to overcome this barrier in public environments of wide concurrence, where much of the infonnation is 
provided to users through oral communication. Therefore, we want to provide a channel of communication that will help 
to improve the integration and independence of this group and contribute to the Universal Accessibility referred in the 
International Convention on the Rights of Persons with Disabilities. 

With this objective to alleviate the problem of communication and facilitate access to infonnation, we propose an 
alternative communication system to “Public Address System, PA ” used in the public areas of wide concurrence, based 
on the translation of oral messages to the SSL, using interpreter a lively 2-dimensional (2D) character. 

For the development of this research, an airport has been chosen, due to the large volume of users that travel daily, 
availability of audiovisual infrastructure and facility to capture messages to translate. 

The videos obtained in 2D, for each message translated into SSL, have been subjected to a validation process by 
members of an association of hearing disabled, who have contributed their opinions and suggestions for improvement, 
which have been applied to the final multimedia application. 
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1. INTRODUCTION 

Within the group of people with hearing disability, many of them use as the first, and in some cases as the 
sole mode of communication, sign language. The purpose of this work is to facilitate access to information in 
environments where audible warning (warning messages, including emergency messages), is essential for 
orientation and adaptation. The intention is to contribute to the independence and integration of these people 
into society, improving their quality of life and compliance with the "Universal Accessibility for people with 
disabilities." 

In the case of persons suffering from hearing disability, it is necessary to place at their disposal all the 
available resources for access to information, "the knowledge and autonomous learning, without which the 
information technology and communication, which can be an opportunity, become a new barrier. "(Utray 
Delgado & Ruiz, 2011). Example of this is at airports, where messages considered of general interest to 
users, do not reach everyone equally. Some people with hearing disability find it difficult and, in many cases 
impossible, to receive those messages. In this sense, in recent years there have been many improvements at 
airports, introducing specialized personnel at points of information or installing induction loops to provide a 
much clearer audible information for everyone using hearing aid or cochlear implant. However, these 
interventions are not completely solving the communication problems encountered by people using only sign 
language as a communication medium. 
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As is well known, "there is no universal sign language" (Ortiz, 2005). Each country has its own sign 
language. Even within each Spanish community there are versions of the language. Due to this circumstance, 
to carry out the application posed, the Spanish sign language has been chosen, declared as official language 
in Law 27/2007 of 23 October (BOE. 255), by which Spanish sign language are recognized and regulate the 
support means for oral communication of deaf people, with hearing disability and deaf blind. Variations 
between the sign languages of the different Spanish regions are not very pronounced, except in Catalonia. 

The public places that may be subject to an action such as that posed include, among others: airports, 
ports, train stations and subway, all kinds of sports facilities, shopping centers and other public facility where 
information is transmitted to users through voice communication. In these places, the information provided 
by PA system, screens and posters may be insufficient to enable such people to cope with complete 
independence as, in addition to not listen to the messages offered by PA system, may have difficulty reading 
written texts. However, it is possible that they can recognize the messages transmitted in sign language. 

Once the various possible scenarios are established, an airport has been chosen for the development of 
this application, due to the ease of capturing messages of PA system, high user concurrency and disposition 
of large spaces where screens can be placed for the issuance of the videos in sign language. 

There are other means of disseminating information, which are increasingly in demand such as interactive 
screens by mobile applications or tablet, via WI-FI. 


2. OBJECTIVES 

The objective of this work consist to make a multimedia application to translate messages advertisements and 
notices issued by PA system and information boards to SSL, through an animated character or avatar, in two 
dimensions (2D). This intervention will allow deaf people to access information with greater autonomy. 

The issuance of these animations can be made by two ways. On the one hand, an unidirectional diffusion 
consisting of storing animations on a server and every so often projected on the screens located on the 
premises for this service, while messages are sent over the PA system. On the other hand, an interactive 
application that allows access to information on flights status in terminals, distributed throughout the 
enclosure. In addition, these devices will emit videos with the information disseminated by PA system 
sequentially to a screensaver, and can be interrupted at one point, said automatic issuance by a user to query 
information about the status of a flight of interest. 

Consequently, this intervention is to take a further step towards the integration of persons with hearing 
disability, trying regardless of the degree of it, either partially or totally, that they can move into a better 
environment adapted to disability, following the signs and messages presented through projections of short 
videos in sign language accompanied by subtitles. 


3. METHODOLOGY 

The performance of this audiovisual application is structured in different phases, which have direct and 
sequential relationship between them: capture of messages, message translation to sign language, character 
design and conversion of video messages captured in 2D animation, implementation of an interactive 
graphical environment for the selection of messages and validation of videos. 

3.1 Capture of Messages 

The types of hearing messages sent in these facilities are diverse. It can be heard recommendations on the use 
of facilities, notices or rules established in the enclosure, in addition to information on the status of flights. 
Information regarding the status of flights can also be found on monitors distributed throughout the 
enclosure. 
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In Spanish airports, managed by the company AENA, messaging is standardized. In this work has been 
chosen Gran Canaria airport for the selection and recording of audio and visual messages. The uptake of the 
messages has been collected in two different ways: Through audio recordings on the airport enclosure and 
through written information posters distributed by the airport. 

3.2 Translation of Messages to Sign Language 

The next step for the realization of the application was to translate the advertisement and notices and / or 
written to the SSL. To carry out this phase of the work it is necessary to have a SSL interpreter responsible 
for this task. 

In the flight number selection messages, included in the notices messages section, small changes will be 
made by reducing the number of digits from six to three, to get a better operability. This allows a more agile 
and entertaining presentation of the resulting application. 

3.3 Character Design and Conversion of Video Messages Captured in 2D 
Animation 

Currently there are many programs for creating vector animations very easily, offering all kinds of tools from 
the most basic to others that require more skill to use, some require licensing fees, while others are 
completely free, although these generally do not have the same benefits. Depending on the level of animation 
you want to achieve, it should be choosen the one that best suits the requirements of the final product. 

At this phase an analysis of the main tools available today was made in the field of computer animation, 
both professional and amateur. The most common vector animation programs where differences can be two 
types, free open source, like Ajax Animator, KToon, Pencil or Synfig, and the ones which have to be payed, 
such as Adobe Flash, Adobe AfterEffects, Anime Studio or Toon Boom, among others. 

After analyzing the different software options and the study of their characteristics, it was assessed what 
was more adapted to the needs and defendants technical requirements, choosing the specific animation 
software known as Adobe Flash CS5, which is an application in the form of study working on "frames" and 
is intended for the production and delivery of interactive content. It uses both vector graphics and raster 
images (raster image or bitmap), sound, program code, video streaming and bidirectional audio. Additionally, 
this program incorporates animation technique called rotoscoping. The advantage of the rotoscoping 
technique is that it replaces people’s faces, which may be more or less attractive, for an animated character. 
By using this technique, the animation vectors were obtained. With these animations and the incorporation of 
some of the elements of the scene, like the stage and wallpaper, it has been achieved the final videos in SSL, 
where the performer becomes an animated character or avatar. Using an avatar is justified by the omission of 
the identity of the performer, it "depersonalizes" him, therefore produces a neutral perception of the message. 

3.4 An Interactive Graphical Environment for the Selection of Messages 

The next phase of work involves the creation of multimedia applications, that the user can select each video 
in SSL. It aims to design a friendly environment that captures the attention of users with interactive 
applications that allow users to interact with items on the screen, such as buttons or text windows. These will 
enter the desired orders by peripherals, which allow to control such interfaces to obtain the requested 
information. The software used for the programming of such applications has been the "ActionScipt 3.0", 
which is the native programming language of Flash. 

According to the objectives, they will be performed by two multimedia applications, which are discussed 
below. 

3.4.1 Application 1: Simulation destined to Video Channel 

From the previous videos, and through appropriate programming, it will be performed a unidirectional 
diffusion, storing animations in a server and each time a message is issued by PA system, simultaneously the 
corresponding video is displayed in SSL on screens distributed on site. 
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3.4.2 Application 2: Simulation of Interactive Application to Query Flight in Electronic 
Devices 

This application has been designed an interactive query system, where the user can check the status of his 
flight by entering the code on an electronic device, which has a keyboard and monitor on which information 
is presented in SSL the state in which it is located. During the time in which no information is requested, on 
the device’s screen will be displayed a screensaver with messages in SSL made for first application. They 
will follow a set pattern of visualization, which will abort only when a user interacts with the device. At this 
point the user is asked to enter flight details he wants to view, to display the information on screen. If after a 
set time it is not interacted with the device, it returns to the screensaver. 

3.5 Validation of the Videos 

After obtaining a first version of the product, it has been subjected to a validation process as a useful tool of 
communication for people with hearing disability. For this, it has been designed a questionnaire with several 
sections that assess technical, aesthetic and intelligibility aspects in communication in SSL. In addition, a 
final section in which an assessment of the application, suggestions and proposals for improvement is sought 
is included. 

Among the videos to validate, four advertisements and four notices messages were chosen. In these 
videos, subtitles and audio were deleted, not to offer any clues about those messages to those who could read 
texts and / or have auditory perception. 

For validation of the videos on SSL, we have been in contact with the Deaf People Association of Gran 
Canaria, who supported us to conduct this phase of the work. From their suggestions, in order to get as many 
samples as possible and have the support of a sign language interpreter to help us communicate with the 
audience, it was made to coincide with the presentation of videos and recollection of surveys, with the 
completion of an event at the headquarters of this Association. 

For the test, it was arranged a room suitable for the purpose. At the beginning of the meeting, the 
attendees were explained the content and objectives of the development work that was to be submitted, with 
the help of interpreter of SSL. Then we proceeded to display videos related with the advertisements in a 42- 
inch LCD monitor, ready for the occasion. Then the questionnaires on these videos were passed to all 
present. Subsequently, the same process was followed with videos concerning notices and related surveys. 

In viewing the videos and conducting surveys, thirteen people participated, nine were women and four 
men, all aged between nineteen and sixty five years. 

To obtain a detailed study of the results, they were analyzed from three different aspects: the collective 
total responses, responses by sex workers and finally, a third analysis separating the responses by age. 


4. RESULTS 

From the methodology discussed, were the results obtained discussed below. 

Four advertisements and four notices messages were chosen. It was considered that this amount is 
sufficient to obtain an objective assessment of the level of acceptance of this work. 

Independent videos were recorded for each of the messages translated to SSL. These videos will be used 
later to make 2D animations to be included in interactive applications. The avatar obtained and the process of 
converting the images of each video to 2D animation is shown in Figure l.The process is repeated for the 
remaining frames of each video. 

In the development of the videos, it has been cared formal aspects like colors fund scene, the character, 
the speed of the gestures, the subtitling or symbols, among others, with the goal of capturing the user’s 
attention. 
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4° Phase 5° Phase 6° Phase 


Figure 1. Changes in a single frame, during the animation process 

The results obtained to the first application are shown in Figures 2, 3 and 4 Figure 2 shows the first screen 
for the main menu, which the user founds when accessing this application. There, has been inserted a 
background image composed of a scenario that is acquainted with the airport environment. For its 
composition, it was used the characters and the logo, taken from a photograph taken outside the enclosure, 
plus the addition of two buttons with the name of their function, which when clicked allows access to the 
submenu. 

When the folder shaped button on the top with an icon showing an exclamation mark is pressed, it 
redirects us to the scene of the advertisement menu, shown in Figure 3.1, where you can choose one of the 
four videos available by pressing the corresponding button. In this section we find six buttons, four for 
displaying the advertisements and two smaller ones, of which one has the function to move the scene of 
notices and the other one returns us to the main screen. 
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Figure 3. Playing a video advertisements menu Figure 4. Playing a video obtained from submenus 

When selecting the screen shown in Figure 2, the button located on the bottom of the main menu, sends 
us to the scene menu notices shown in Figure 4, where you can select messages with information regarding to 
flight status. In this section we find eight buttons, five of which have special behavior, because they contain 
submenus. 

The buttons of these submenus are responsible for selecting the combination of videos you want to 
display. We talked about combining videos and not watching a video, because at this stage you have to do a 
combination of five of them to obtain the final video to be played. This is because in each submenu you have 
to select a message: initial message, each of the digits of the flight and final message. Within each submenu 
there are different videos, as many as messages we can choose in it. In the submenu flight code, the virtual 
keyboard that opens when you click on each of the three buttons shown, all the letters of these submenus are 
associated with their corresponding video of the alphabet and numbers. 
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Once you have selected the corresponding digit on the flight, and have chosen the option of flight status, 
we will have available the "play" button to play the message indicating, in sign language, flight status with 
the typed code, such as shown in Figure 4. 

For "Application 2: Simulation of interactive application for flight query in electronic devices " has 
designed the screensaver system shown in Figure 4, which the user encounters when accessing this 
application. All scenes are formed under one animation pattern and is made of two elements. Initially it is 
displayed the back of the stage and an animation that is shown as a text that scrolls across the screen from 
right to left showing the message : "PRESS ANY KEY TO ENTER YOUR FLIGHT NUMBER." With it, it 
is intended that the user should be aware that act on the keyboard to get the information you want. When you 
finish the text displayed on the monitor, it starts playing one of the videos that have been animated before. 



Figure 5. Scenes and videos that make up the screen saver 

We have added new elements to these videos and that belong to the symbols of the folders in the first 
application. To display on this application, it was decided to change the scale and color, choosing in this case 
red. They have also been provided with a small animation, creating a flashing effect. This is to reinforce, in a 
subtle way, the information that is being offered to the user. 

When a user interacts with the device, he will be shown a screen on their requested flight code. After 
entering this code, the corresponding video on the state of the same is shown, but in the event that the flight 
number is entered wrongly, dynamic text is activated in the center of the screen with the following message: 
"THE FLIGHT NUMBER MARKED IS NOT AVAILABLE." 

For this application to work properly it is necessary that the interactive device has updated information on 
identification data and flight status, the same way that the information provided by other means such as 
information screens and PA system is renewed. It is necessary that the information in all media is 
simultaneously updated. This task can be performed automatically by connecting the device to the 
corresponding database or manually, in which an operator is responsible for updating this information. 

Once the results shown for both applications, the results of the evaluations obtained through surveys are 
presented. 

In relation to the compression of the videos, it can be seen that about half of the attendees suggests they 
can be improved. The results for questions related to the compression of the videos, are shown in Figure 6. 


ADVERTISEMENTS NOTICES 

VERY 
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Figure 6. Level of satisfaction with the videos shown 
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Regarding the use of a character or avatar in 2D, 84% of attendees were in favor. If we analyze by gender, 
we see that every men and 76% of women liked the video presented with a 2D character as an interpreter of 
SSL. In the results we see that all ages prefer 2D animation, but in the age group of between twenty and 
thirty-five, the acceptance level decreases. Of the three members of this group, each have a different opinion 
to the use of an animated character. 

Regarding the use of subtitles in the videos, 73% of the participants are in favor of its use. In the analysis 
of the responses to this question by gender, we see that the vast majority of both sexes considered appropriate 
to include subtitles to the videos that have been shown to them. In the study based on the responses of 
different age, the same general trend of preference subtitles in the videos is perceived. However, it should be 
pointed out that 50% of the people in the age range of forty-six to sixty have failed to answer this question. 

Finally, an assessment is requested for the videos displayed on a scale of 1 to 10, obtaining the results 
shown in the following table. 


Table 1 . Overall rating of the videos shown 


GLOBAL ASSESSMENT OF VIDEOS ON SSL 

Advertisements 

Notices 

Rated Result Percentage 

Result Percentage 


1...4 

0 

0% 

0 

0% 

5 

1 

8% 

2 

15% 

6 

2 

15% 

0 

0% 

7 

1 

8% 

0 

0% 

8 

1 

8% 

2 

15% 

9 

1 

8% 

1 

8% 

10 

7 

54% 

8 

62% 

NULL 

0 

0% 

0 

0% 

SIGNS 

13 

100% 

13 

100% 


5. CONCLUSIONS 

This article has presented a multimedia product development that can extend the possibilities of 
communication in public places with people who use the Spanish sign language. This communication system 
can be particularly useful to provide people with hearing disability an avenue of further communication, 
contributing to an increase in personal autonomy. It also has the advantage that this model can be exported to 
any public setting where required, to convey information to users. 

The environment object to the simulated study was an airport. The results show that an application of this 
characteristics and minimal cost can provide a broader communication service based sector of the population 
who may have difficulty accessing messaging and sound alerts. In this regard, it is particularly interesting 
that no additional infrastructure is required to present at sites of public assembly. 

In addition, it has been shown that a similar submitted development provides the ability to offer users a 
friendly multimedia simulation, placed in strategic places for information points, with the aim of expanding 
the communication channels available to inform passengers’ flight status and other information that may be 
of interest. 

In particular, the use of the technique of rotoscoping to create the avatar appears to provide a valid and 
depersonalized approach to sign language translation using a human figure. 

Although the number of samples in the validation survey is small and it is necessary, therefore, a larger 
study, we can conclude that the majority of user perception is positive, although messages can be improved. 
In this sense, our hypothesis is corroborated, in which we proposed that it should be added subtitled messages 
to the videos, thereby achieving a more complete communication. The validation results also indicate the 
desirability of improving the character animation, especially in his body language. 

This work is a pilot test to assess the use of rotoscoping and animation technique to create a 
communicative environment improvement for the hearing disability. This model and workflow allows the 
extension of the results to the official sign languages of other countries and environments. Additionally, it 
can be used other devices that allow the reception of messages such as tablets and mobile phones. 
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The experience obtained in this work can be transferred to other fields such as education, in which has 
utility as an augmentative communication system for hearing disability people. In this sense, it may be 
interesting to incorporate a virtual interpreter in the class that repeat the oral explanation of the teacher, in 
sign language. 
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